Learning device and method

ABSTRACT

A method and device includes an input/output interface, a memory, and one or more processors communicably coupled to the input/output interface and the memory. The input/output interface includes a display and a speaker. A musical or linguistic work is selected via the input/output interface. A visual data and an audio data for the selected musical or linguistic work is retrieved from the memory or the input/output interface using the one or more processors. All or part of the visual data is displayed on the display. A portion of the displayed visual data is selected via the input/output interface. A portion of the audio data corresponding to the selected portion of the visual data is played using the speaker and the one or more processors.

CROSS-REFERENCE TO RELATED APPLICATIONS

Not applicable.

STATEMENT OF FEDERALLY FUNDED RESEARCH

This application is not the subject of any federally sponsored researchor development.

TECHNICAL FIELD OF THE INVENTION

The present invention relates in general to the field of education andtraining, and more particularly, to a learning device and method.

BACKGROUND OF THE INVENTION

Without limiting the scope of the invention, its background is describedin connection with learning music and languages.

Music educators have long encountered a problem that plagues studentsand parents. Students currently do not have the tools to practicewithout errors when they go home. Students practice errors that becomeconcrete and buried in their subconscious mind making it difficult tocorrect in the next lesson. The vast majority of music anthologies comewith a companion audio with the intention of helping students practiceboth aurally and visually, but the two are currently disconnected.

Conventional learning tools and sources of instruction for learningmusic include music teachers, music books, method books, anthologies,compact disks (CDs), digital video disks (DVDs) and MP3 downloadableaudio files. While each of these offers certain advantages, each alsohas limitations. The audio and written learning resources have thedisadvantage of the quality of instruction and the manner in which theinformation is presented. Highly valued teachers are exceedingly limitedin the amount of time spent with each student. For example, a musicteacher provides a personally tailored lesson; however, the cost of theinstruction is expensive and each session is limited not just in timebut based on several factors. One of the main factors is the students'ability to retain information from lesson to lesson. The vast majorityof private lessons are spent reviewing the previous material taught theprevious week and depending on the student, material from several weeksprior. This is due, primarily, to the practice regimen not beingconsistent at home. They may have the desire to practice, or they mayeven be practicing but they are doing so incorrectly which activelymakes them regress rather than progress. In addition, the majority ofparents are not musicians themselves and do not have the knowledge orskill set to help their children practice their particular instrument.To develop proficiency, the student must devote many hours each week topracticing the assigned material individually. Music books provide arelatively low-cost alternative or adjunct to a music teacher. Most ofthe newer music books include companion audio in addition to the notatedmusic and lessons, which will usually include some general instruction.

MP3 or compact disks combine recorded audio with instructionalcommentary and helpful “hints” for performance. Some audio includessongs and musical passages recorded both at a normal tempo and reducedtempo to make learning easier. With both MP3 and CDs however, looping orrepeating a particular section of a song is cumbersome and difficult.The ability to stop a CD player, that is if you can still purchase a CDplayer, and go directly to that passage is very hard and slows down thelearner. Not to mention the student is already frustrated with theproblematic passage and now the audio intended to help him or her iscomplicating the learning process even further. In addition, if thesongs and musical passages are not recorded at a reduced tempo thenreducing the tempo using a conventional CD player is not possible.Moreover, CDs and MP3s do not provide any visual instruction orinteraction, which makes it challenging for the student to develop aclear aural/visual understanding of the music.

Currently, most methods of language teaching are provided in the form oftraditional textbooks that focus on teaching grammar and vocabulary.Although such textbooks provide a general knowledge of the structure ofthe new language, a major difficulty for the language learners is makinguse of this knowledge in real-life situations. Crucially, individuals donot learn their native languages by studying grammatical structures ormemorizing lists of words, but by being exposed to language use inauthentic communicative settings. Moreover, children learn their nativelanguages orally, long before they start learning how to read.Therefore, purely text-based instructions are considered insufficientfor gaining native-like mastery in a second language. Additionally, manysecond language learners do not have access to native-speaking teacherswho can provide them with correct pronunciation and usage tips. Beingexposed to incorrect or inaccurate instructions by non-native teacherscan lead to fossilization of errors, which would be very difficult tocorrect later. Even when native-speaking teachers are available, thestudent's access to them is normally limited only to the classroom time.Since students do not have access to correct usage of what they havelearned when they are outside of the classroom, it is very likely thatthey spend time practicing incorrect forms, which would require extendedtime and effort to correct. While some language textbooks try to resolvethis problem by providing audio CDs or MP3 files of the texts spoken bynative speakers, having the texts and the audio files in differentmediums would make these audio files difficult or, in some situations,impossible to use.

SUMMARY OF THE INVENTION

The various embodiments of the invention disclosed herein provide a newand improved way for people to learn to play or sing music, and learnlanguages. The combination of audio, visual and sheet music enablespeople to associate the sounds with the notation, which enhanceslearning. Likewise, the combination of audio, visual and text enablespeople to associate the sounds with the words and/or musical notes,which enhances learning. This innovative and interactive learningsoftware aids music students in proper and effective practice of amusical instrument and/or vocal studies.

In one embodiment, a device includes an input/output interface, amemory, and one or more processors communicably coupled to theinput/output interface and the memory. The input/output interfaceincludes a display and a speaker. A musical or linguistic work isselected via the input/output interface, and a visual data and an audiodata for the selected musical or linguistic work is retrieved from thememory or the input/output interface using the one or more processors.All or part of the visual data is displayed on the display. A segment ofthe displayed visual data is selected via the input/output interface,and the segment of the audio data corresponding to the selected segmentof the visual data is played using the speaker and the one or moreprocessors.

In one aspect, the visual data comprises a musical score data for themusical work, and the visual data comprises a text, character, gesture,sign and/or symbol data for the linguistic work. In another aspect, thevisual data and the audio data are stored together in one or more filesor stored separately in the one or more files. In another aspect, aspecified period of time is waited and the segment of the audio datacorresponding to the selected segment of the musical score data isreplayed using the speaker and the one or more processors. In anotheraspect, the specified period of time and a specified number of replaysare selected via the input/output interface. In another aspect, amusical instrument, a vocal part, or a language is selected using theinput/output interface, wherein the visual data and the audio datacorrespond to the selected musical instrument, the selected vocal part,or the selected language. In another aspect, an indication is providedon the display corresponding to the audio data being played using theone or more processors.

In another aspect, a user rendition of the selected segment of thevisual is recorded using the input/output interface. In another aspect,the user rendition is compared to the audio data, an accuracy score forthe user rendition is calculated, and the accuracy score is displayed onthe display using the one or more processors. In another aspect, one ormore suggestions to improve the accuracy score are provided on thedisplay using the one or more processors. In another aspect, one or moreerrors in the user rendition are displayed or highlighted on the displayusing the one or more processors. In another aspect, a user visual datais created based on the user rendition and the rendition is displayed onthe display using the one or more processors. In another aspect, anydifferences in the created visual data and the visual data arehighlighted or visually indicated on the display using the one or moreprocessors. In another aspect, an animation or video associated with thevisual data is displayed on the display using the one or moreprocessors.

In another embodiment, a method for musical or linguistic learningincludes providing a device that includes an input/output interface, amemory and one or more processors communicably coupled to theinput/output interface and the memory. The input/output interfaceincludes at least a display and a speaker. A musical or linguistic workis selected via the input/output interface. A visual data and an audiodata for the selected musical or linguistic work is retrieved from thememory or the input/output interface using the one or more processors.All or part of the visual data is displayed on the display. A segment ofthe displayed visual data is selected via the input/output interface. Asegment of the audio data corresponding to the selected segment of thevisual data is played using the speaker and the one or more processors.Optionally, a specified period of time is waited and the segment of theaudio data corresponding to the selected segment of the visual data isreplayed using the speaker and the one or more processors.

In one aspect, the visual data comprises a musical score data for themusical work, and the visual data comprises a text, character, gesture,sign and/or symbol data for the linguistic work. In another aspect, thevisual data and the audio data are stored together in one or more filesor stored separately in the one or more files. In another aspect, themethod further comprises selecting the specified period of time and aspecified number of replays via the input/output interface. In anotheraspect, the method further comprises selecting a musical instrument, avocal part, or a language using the input/output interface, wherein thevisual data and the audio data correspond to the selected musicalinstrument, the selected vocal part, or the selected language. Inanother aspect, the method further comprises providing an indication onthe display corresponding to the audio data being played using the oneor more processors.

In another aspect, the method further comprises recording a userrendition of the selected segment of the visual data using theinput/output interface. In another aspect, the method further comprisescomparing the user rendition to the audio data, calculating an accuracyscore for the user rendition, and displaying the accuracy score on thedisplay using the one or more processors. In another aspect, the methodfurther comprises providing one or more suggestions to improve theaccuracy score on the display using the one or more processors. Inanother aspect, the method further comprises displaying or highlightingone or more errors in the user rendition on the display using the one ormore processors. In another aspect, the method further comprisescreating a user visual data based on the user rendition and displayingthe rendition on the display using the one or more processors. Inanother aspect, the method further comprises highlighting or visuallyindicating any differences in the user visual data and the visual dataon the display using the one or more processors. In another aspect, themethod further comprises displaying an animation or video associatedwith the musical work on the display using the one or more processors.In another aspect, the method further comprises performing the selectedsegment of the visual data by a user. In another aspect, performing theselected segment of the visual data by the user occurs after or incombination with playing the portion of the audio data corresponding tothe selected portion of the visual data.

In one embodiment, the software can be used as a method of providingreal-life examples of conversations in a second language in specificcommunicative contexts, along with the ability to repeat individualwords, phrases and sentences. The text of a dialogue between two or morepeople in a specific situation (e.g., a conversation between a doctorand a patient, or between a bank teller and a customer) is presented onthe display. A segment of the displayed text is selected via theinput/output interface, and the segment of the audio data correspondingto the selected segment of the text is played using the speaker and theone or more processors. In one aspect, a specified period of time iswaited and the segment of the audio data corresponding to the selectedsegment of the text is replayed using the speaker and the one or moreprocessors. In another aspect, the specified period of time and aspecified number of replays are selected via the input/output interface.In another aspect, a word, phrase or sentence is selected using theinput/output interface, wherein the text data and the audio datacorrespond to the selected word, phrase or sentence. Instructionalinformation or tips can also be displayed, including the author'srecorded voice coaching on specific parts of the text or general usagepoints.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the features and advantages of thepresent invention, reference is now made to the detailed description ofthe invention along with the accompanying figures in which:

FIG. 1 is an image of a screen displayed in accordance with oneembodiment of the present invention;

FIGS. 2A-2D are images of screens highlighting various features of oneembodiment of the present invention;

FIG. 3 is a block diagram of an apparatus in accordance with oneembodiment of the present invention;

FIGS. 4A-4B are flow chart of a method in accordance with one embodimentof the present invention;

FIG. 5 is an image of a screen displayed in accordance with oneembodiment of the present invention; and

FIGS. 6A-6D are images of screens highlighting various features of oneembodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Illustrative embodiments of the system of the present application aredescribed below. In the interest of clarity, not all features of anactual implementation are described in this specification. It will, ofcourse be appreciated that in the development of any such actualembodiment, numerous implementation-specific decisions must be made toachieve the developer's specific goals, such as compliance withsystem-related and business-related constraints, which will vary fromone implementation to another. Moreover, it will be appreciated thatsuch a development effort might be complex and time-consuming but wouldnevertheless be a routine undertaking for those of ordinary skill in theart having the benefit of this disclosure.

In the specification, reference may be made to the spatial relationshipsbetween various components and to the spatial orientation of variousaspects of components as the devices and systems are depicted in theattached drawings. However, as will be recognized by those skilled inthe art after a complete reading of the present application, thedevices, systems, members, apparatuses, etc. described herein may bepositioned in any desired orientation. Thus, the use of terms such as“above,” “below,” “upper,” “lower,” or other like terms to describe aspatial relationship between various components or to describe thespatial orientation of aspects of such components should be understoodto describe a relative relationship between the components or a spatialorientation of aspects of such components, respectively, as the devicesand systems described herein may be oriented in any desired direction.

Now referring to FIG. 1, an image of a screen displayed in accordancewith one embodiment of the present invention is shown. The functionsdescribed herein pertain to the embodiment shown, but those skilled inthe art will recognize that many other functions can be integrated inthe present invention. Some of these other functions will be describedbelow. The embodiment shown is displayed on an electronic tablet.However, any mobile communications device or computing device can beused, such as a mobile phone, laptop computer, desktop computer, smartTV, smart glasses, gaming device, etc. The present invention iscompatible with any general platform, mac, pc, Android, iPhone, etc.Although a connection to the Internet via the input/output interface ispreferred, some embodiments may to require such a connection. Theinput/output interface can be any combination of touch display,keyboard, mouse, voice control, motion control, a wired communicationsinterface, a wireless communications interfaces, or other way to receivedata/commands or provide data/information to a user.

A musical work can be selected from the Library button 102. The musicalwork can be a single musical work, a set of musical works, an educationor training program, or other compilation. Moreover, the musical workcan be all or part of a complete musical work (e.g., song, chant,instrumental, etc.). In this embodiment, the musical work is a guitartraining program. The musical work can be stored locally in memory or astorage medium, or remotely (e.g., a server accessible via a wirelessconnection, etc.). Once selected, information about the musical workwill be displayed in the main display area 104, such images of guitarsidentifying the various parts of the guitar. In other embodiments, themain display area 104 may include a title, author, copyrightinformation, graphics, etc. An action or function bar 106 includes aplay/pause button 108, a loop button 110, an information area 112, alocation icon 114, a full screen icon 116 and a settings button 118.Other buttons, icons and information can be displayed. If the selectedwork is divided into parts, chapters, lessons, tasks, etc., they will bedisplayed in the navigation bar 120. In this example, the name of themusical work “First Steps in Guitar” is displayed in the informationarea 112 and the current location “1” within the navigation bar 120 isdisplayed in the location icon 114. The navigation bar 120 includesbuttons that allow the user to jump to a specific location within themusical work. In this example, the navigation buttons include “The Partsof Your Guitar” 122, “Tuning” 124, “Playing Position” 126, “How to ReadMusic and Tab” 128, “Scales” 130, “Yankee Doodle” 132, “First Chords(Key of G)” 134, “4/4 Basic Strum” 136, “3/4 Basic Strum” 138, “OneOctave Open G Scale” 140, “One Octave Open C Scale” 142, and “Down & UpStrokes” 144.

Referring now to FIGS. 2A-2D, images of screens highlighting variousfeatures of one embodiment of the present invention are shown. Thenavigation button 130 for the “Scales” is selected in FIG. 2A. Themusical score data 202 is displayed in the main display area 104, thenavigation button 130 “Scales” is highlighted, and the current location“5” is displayed in the location icon 114 in FIG. 2B. Musicalinformation 204 and graphics or video 206, which can be interactive oranimated, are also displayed in the main display area 104. Instructionalinformation or tips can also be displayed, including the author'srecorded voice coaching on the difficult sections of the music. Forexample, information about “Scales” is displayed along with fretboarddiagrams for the tones in each scale. In this case, the musical scoredata 202 includes a musical score and a tablature (TAB). In otherembodiments, the musical score data 202 may include lyrics. The musicalscore 202 can be segmented into segments, phrases or portions to assistin the learning or training process (e.g., segments 202 a, 202 b, 202 c,202 d, 202 e and 202 f). The user can select one of the segments orselect their own portion of the musical score 202. In this example, theuser selects score segment 202 b, which is highlighted while playing, asshown in FIG. 2C. The play button 108 switches to a pause button whenthe musical selection is being played, and switches back to a playbutton when the musical selection if paused. As shown in FIG. 2D, asettings bar 210 is displayed when the settings button 118 is selected.The settings bar 210 includes a delay data box 212 “Wait for seconds”, anumber of repetitions data box 214 “Segment repeat”, an autoplay on/offslide button 216, a reset button 218, a submit button 220, and a cancelbutton 222.

Students can select any phrase they would like to play and echo back,similar to modeling provided by their teacher. The selected phrase willbe played and a custom pause time will be observed giving the user timeto echo it back on their instrument or voice. This creates call andresponse learning. The user can customize the number of repetitions andthe seconds in between each phrase.

Although not required, it is preferred that all recordings are actualvoice and/or instruments, which conveys artistry and serves as a correctmodel of musical tone production (no synthesized or midi sounds areused). As a result, students can hear the actual audio, not a Midisound, by the performer and visually see the musical phrases on thesheet music. This option of repetition can be used as many times as theywant. This helps students easily associate the written notation or awritten melody to the sound, reinforcing what is taught in privatelessons or music class/band/choir/orchestra.

The present invention complements what educators do in the classroom orstudio, and serves as a companion aid for students at home, and helpfulnew resources for teachers. The student's practice time can be reported,which adds accountability for teachers and parents. Moreover, in someembodiments, the student's performances can be recorded, which addsaccountability for teachers and parents. The present invention can beused any instrument, voice, or language. Existing books, CDs, or MP3scan be easily converted utilizing the resources already invested tocreate these media. In some embodiments, the present invention can beused in a game mode were individuals compete against one another locallyor via an Internet connection.

Now referring to FIG. 3, a block diagram of a device 300 in accordancewith one embodiment of the present invention is shown. The device 300can be an electronic tablet, a mobile phone, a laptop computer, adesktop computer, a smart TV, a smart glasses, a gaming device, or anyother device capable of performing the functions described herein. Thedevice 300 includes an input/output interface 302, a memory 304, and oneor more processors 306 communicably coupled to the input/outputinterface 302 and the memory 304. Note that the device 300 may includeother components not specifically described herein. The memory 304 canbe local, remote or distributed. Likewise, the one or more processors306 can be local, remote or distributed. The input/output interface 302includes a display 308 and a speaker 310. The input/output interface 302can be any mechanism for facilitating the input and/or output ofinformation (e.g., touch display, keyboard, mouse, voice control, motioncontrol, a wired communications interface, a wireless communicationsinterfaces, or other way to receive data/commands or providedata/information to a user, etc.) Moreover, the input/output interface302 can be a remote device communicably coupled to the one or moreprocessors 306 via one or more communication links 312 (e.g.,network(s), cable(s), wireless, satellite, etc.). The one or morecommunication links 312 can communicably couple the device 300 to otherdevices 314 (e.g., databases, servers, other similar devices ofstudents, teachers and/or performers, etc.). A musical work is selectedvia the input/output interface 302, and a musical score data and anaudio data for the selected musical work is retrieved from the memory304 or the input/output interface 302 using the one or more processors306. All or part of the musical score data is displayed on the display308. A segment of the displayed musical score data is selected via theinput/output interface 302, and the segment of the audio datacorresponding to the selected segment of the musical score data isplayed using the speaker 310 and the one or more processors 306.

In one aspect, the visual data comprises a musical score data for themusical work, and the visual data comprises a text, character, gesture,sign and/or symbol data for the linguistic work. In another aspect, thevisual data and the audio data are stored together in one or more filesor stored separately in the one or more files. In another aspect, aspecified period of time is waited and the segment of the audio datacorresponding to the selected segment of the visual data is replayedusing the speaker 310 and the one or more processors 306. In anotheraspect, the specified period of time and a specified number of replaysare selected via the input/output interface 302. In another aspect, amusical instrument, a vocal part or a language is selected using theinput/output interface 302, wherein the visual data and the audio datacorrespond to the selected musical instrument, the selected vocal part,or the selected language. In another aspect, an indication is providedon the display 308 corresponding to the audio data being played usingthe one or more processors 306.

In another aspect, a user rendition of the selected segment of themusical score is recorded using the input/output interface 302. Inanother aspect, the user rendition is compared to the audio data, anaccuracy score for the user rendition is calculated, and the accuracyscore is displayed on the display 308 using the one or more processors306. In another aspect, one or more suggestions to improve the accuracyscore are provided on the display 308 using the one or more processors306. In another aspect, one or more errors in the user rendition aredisplayed or highlighted on the display 308 using the one or moreprocessors 306. In another aspect, a user visual data is created basedon the user rendition and the rendition is displayed on the display 308using the one or more processors 306. In another aspect, any differencesin the user visual data and the musical score are highlighted orvisually indicated on the display 308 using the one or more processors306. In another aspect, an animation or video associated with the visualdata is displayed on the display 308 using the one or more processors306.

Referring now to FIGS. 4A-4B, flow charts 400 of a method in accordancewith one embodiment of the present invention is shown. A device includesan input/output interface, a memory and one or more processorscommunicably coupled to the input/output interface and the memory isprovided in block 402. The input/output interface includes at least adisplay and a speaker. A musical or linguistic work is selected via theinput/output interface in block 404. A visual data and an audio data forthe selected musical or linguistic work is retrieved from the memory orthe input/output interface using the one or more processors in block406. All or part of the visual data is displayed on the display in block408. A segment of the displayed visual data is selected via theinput/output interface in block 410. A segment of the audio datacorresponding to the selected segment of the visual data is played usingthe speaker and the one or more processors in block 412. At this point,the user will play, sing, speak, write or otherwise perform the selectedsegment of the visual data in block 414. Alternatively, the user canattempt to play, sing, speak, write or otherwise perform the selectedsegment of the visual data simultaneously with the playing of theselected segment in block 412, which can assist the user in performingthe correct timing, rhythm, tone or pitch of the selected segment of thevisual data. A user or the user's teacher may select various options orfeatures to assist in the learning process. Default selections for thevarious options or features may also be used.

If any options or features are selected or enabled, as determined indecision block 416, the process will perform the options or features asillustrated in FIG. 4B. The invention is not limited to the options orfeatures shown, or the order in which they are shown. If a record optionis selected, as determined in decision block 418, the performance of theselected portion of the visual data by the user is recorded in block420. Note that this option would be performed prior to or concurrentlywith the user's performance in block 414. If a playback option isselected, as determined in decision block 422, the recording is playedin block 424. If the playback option is not selected, as determined indecision block 422, or the after the recording has been played in block424, or the record option is not selected, as determined in decisionblock 418, the next option is evaluated. If a metrics option isselected, as determined in decision block 426, the one or more metricsare computed in block 428. The metrics can include, but is not limitedto, any data regarding the use by the user (e.g., identification, timebegan, selections, repetitions, time end, etc.), or analysis of theuser's performance (e.g., accuracy, automated critic, automatedsuggestions, etc.). Any other type of data or analytics can be used. Ifthe metrics are to be transmitted to a server, teacher or other device,as determined in decision block 430, the metrics are transmitted inblock 432. If the transmit option is not selected, as determined indecision block 430, or the after the metrics have been transmitted inblock 432, or the metrics option is not selected, as determined indecision block 426, the next option is evaluated. If the replay optionhas been selected and there are replays left to be played, as determinedin decision block 434, a specified period of time is waiting in block436, the segment of the audio data corresponding to the selected segmentof the visual data is replayed using the speaker and the one or moreprocessors in block 412, and the process continues as described.Alternatively, the device can wait for the user to play, sing, speak,write or otherwise perform the selected segment of the visual data usingaudio sensors (e.g., microphone, etc.) and/or visual sensors (e.g.,camera, etc.) before automatically replaying the selected segment of thevisual data in block 412. If the replay option has not been selected orthere are no replays left, as determined in decision block 434, or nooptions are enabled, as determined in decision block 416, the processends in block 438. Note that in some cases, some steps can be reordered,altered or eliminated. Moreover, other steps can be added.

In one aspect, the visual data comprises a musical score data for themusical work, and the visual data comprises a text, character, gesture,sign and/or symbol data for the linguistic work. In another aspect, thevisual data and the audio data are stored together in one or more filesor stored separately in the one or more files. In another aspect, themethod further comprises selecting the specified period of time and aspecified number of replays via the input/output interface. In anotheraspect, the method further comprises selecting a musical instrument, avocal part, or a language using the input/output interface, wherein thevisual data and the audio data correspond to the selected musicalinstrument, the selected vocal part, or the selected language. Inanother aspect, the method further comprises providing an indication onthe display corresponding to the audio data being played using the oneor more processors.

In another aspect, the method further comprises recording a userrendition of the selected segment of the musical score using theinput/output interface. In another aspect, the method further comprisescomparing the user rendition to the audio data, calculating an accuracyscore for the user rendition, and displaying the accuracy score on thedisplay using the one or more processors. In another aspect, the methodfurther comprises providing one or more suggestions to improve theaccuracy score on the display using the one or more processors. Inanother aspect, the method further comprises displaying or highlightingone or more errors in the user rendition on the display using the one ormore processors. In another aspect, the method further comprisescreating a user visual data based on the user rendition and displayingthe rendition on the display using the one or more processors. Inanother aspect, the method further comprises highlighting or visuallyindicating any differences in the user visual data and the visual dataon the display using the one or more processors. In another aspect, themethod further comprises displaying an animation or video associatedwith the musical work on the display using the one or more processors.In another aspect, the method further comprises performing the selectedsegment of the visual data by a user. In another aspect, performing theselected segment of the visual data by the user occurs after or incombination with playing the portion of the audio data corresponding tothe selected portion of the visual data.

Now referring to FIG. 5, an image of a screen displayed in accordancewith one embodiment of the present invention is shown. The software canbe used as a method of providing real-life examples of conversations ina second language in specific communicative contexts, along with theability to repeat individual words, phrases and sentences. The text of adialogue between two or more people in a specific situation (e.g., aconversation between a doctor and a patient, or between a bank tellerand a customer) is presented on the display. A segment of the displayedtext is selected via the input/output interface, and the segment of theaudio data corresponding to the selected segment of the text is playedusing the speaker and the one or more processors. In one aspect, aspecified period of time is waited and the segment of the audio datacorresponding to the selected segment of the text is replayed using thespeaker and the one or more processors. In another aspect, the specifiedperiod of time and a specified number of replays are selected via theinput/output interface. In another aspect, a word, phrase or sentence isselected using the input/output interface, wherein the text data and theaudio data correspond to the selected word, phrase or sentence.Instructional information or tips can also be displayed, including theauthor's recorded voice coaching on specific parts of the text orgeneral usage points.

More specifically, a lesson can be selected from the Library button 502.Once selected, information about the lesson will be displayed in themain display area 504, such as a title, author, copyright information,graphics, etc. An action or function bar 506 includes a play/pausebutton 508, a loop segment button 510, an information area 512, alocation icon 514, a full screen icon 516 and a settings button 518.Other buttons, icons and information can be displayed. If the selectedwork is divided into parts, chapters, lessons, tasks, etc., they will bedisplayed in the navigation bar 520. In this example, the name of thelesson “Personal Services” is displayed in the information area 512 andthe current location “1” within the navigation bar 520 is displayed inthe location icon 514. The navigation bar 520 includes buttons thatallow the user to jump to a specific location within the dialogues. Inthis example, the navigation buttons include “Bank Office” 522, “GroceryStore” 524, “Department Store” 526, “Store Checkout” 528, “Fast FoodRestaurant” 530, “Barbershop” 532, “Pharmacy” 534, “Auto-mechanic Store”536, “Laundry” 538, “Dry Cleaning” 540, “Cell Phone Store” 542, and“Sports Bar” 544.

Referring now to FIGS. 6A-6D, images of screens highlighting variousfeatures of one embodiment of the present invention are shown. Thenavigation button 530 for the “Fast Food Restaurant” is selected in FIG.6A. The dialogue text 602 is displayed in the main display area 504, thenavigation button 530 “Fast Food Restaurant” is highlighted, and thecurrent location “5” is displayed in the location icon 514 in FIG. 6B.Dialogue information 604 and graphics or video 606, which can beinteractive or animated, are also displayed in the main display area504. Instructional information or tips can also be displayed, includingthe author's recorded voice coaching on the difficult sections orimportant usage points in the dialogue. The dialogue text 602 can besegmented into words, phrases or sentences to assist in the learning ortraining process (e.g., segments 602 a, 602 b, 602 c, 602 d, 602 e, 602f, 602 g, 602 h, 602 i, 602 j, 602 k, 602 l, 602 m, 602 n, 602 o and 602p). The user can select one of the segments. In this example, the userselects segment 602 b, which is highlighted while playing, as shown inFIG. 6C. The play button 508 is used to play the selected segment 602 bof the dialogue text 602. The play button 508 switches to a pause buttonwhen the selected segment of the dialogue is being played. As shown inFIG. 6D, a settings bar 610 is displayed when the settings button 518 isselected. The settings bar 610 includes a delay data box 612 “Wait forseconds”, a number of repetitions data box 614 “Segment repeat”, anautoplay on/off slide button 616, a reset button 618, a submit button620, and a cancel button 622.

Students can select any word, phrase or sentence they would like to playand echo back, similar to modeling provided by their teacher. Theselected segment will be played and a custom pause time will be observedgiving the user time to echo it back. This creates call and responselearning. The user can customize the number of repetitions and theseconds in between each phrase.

The embodiment shown in FIGS. 5 and 6A-6D can be implemented with thedevice of FIG. 3 and the method of FIG. 4.

It will be understood that particular embodiments described herein areshown by way of illustration and not as limitations of the invention.The principal features of this invention can be employed in variousembodiments without departing from the scope of the invention. Thoseskilled in the art will recognize, or be able to ascertain using no morethan routine experimentation, numerous equivalents to the specificprocedures described herein. Such equivalents are considered to bewithin the scope of this invention and are covered by the claims.

All publications and patent applications mentioned in the specificationare indicative of the level of skill of those skilled in the art towhich this invention pertains. All publications and patent applicationsare herein incorporated by reference to the same extent as if eachindividual publication or patent application was specifically andindividually indicated to be incorporated by reference.

The use of the word “a” or “an” when used in conjunction with the term“comprising” in the claims and/or the specification may mean “one,” butit is also consistent with the meaning of “one or more,” “at least one,”and “one or more than one.” The use of the term “or” in the claims isused to mean “and/or” unless explicitly indicated to refer toalternatives only or the alternatives are mutually exclusive, althoughthe disclosure supports a definition that refers to only alternativesand “and/or.” Throughout this application, the term “about” is used toindicate that a value includes the inherent variation of error for thedevice or system, the method being employed to determine the value, orthe variation that exists among the fastener subjects.

As used in this specification and claim(s), the words “comprising” (andany form of comprising, such as “comprise” and “comprises”), “having”(and any form of having, such as “have” and “has”), “including” (and anyform of including, such as “includes” and “include”) or “containing”(and any form of containing, such as “contains” and “contain”) areinclusive or open-ended and do not exclude additional, unrecitedelements or method steps. In embodiments of any of the compositions andmethods provided herein, “comprising” may be replaced with “consistingessentially of” or “consisting of.” As used herein, the phrase“consisting essentially of” requires the specified integer(s) or stepsas well as those that do not materially affect the character or functionof the claimed invention. As used herein, the term “consisting” is usedto indicate the presence of the recited integer (e.g., a feature, anelement, a characteristic, a property, a method/process step, or alimitation) or group of integers (e.g., feature(s), element(s),characteristic(s), property(ies), method/process step(s), orlimitation(s)) only.

As used herein, the term “or combinations thereof” refers to allpermutations and combinations of the listed items preceding the term.For example, “A, B, C, or combinations thereof” is intended to includeat least one of: A, B, C, AB, AC, BC, or ABC, and if order is importantin a particular context, also BA, CA, CB, CBA, BCA, ACB, BAC, or CAB.

Continuing with this example, expressly included are combinations thatcontain repeats of one or more item or term, such as BB, AAA, AB, BBC,AAABCCCC, CBBAAA, CABABB, and so forth. The skilled artisan willunderstand that typically there is no limit on the number of items orterms in any combination, unless otherwise apparent from the context.

As used herein, words of approximation such as, without limitation,“about,” “substantial,” or “substantially,” refers to a condition thatwhen so modified is understood to not necessarily be absolute or perfectbut would be considered close enough to those of ordinary skill in theart to warrant designating the condition as being present. The extent towhich the description may vary will depend on how great a change can beinstituted and still have one of ordinary skill in the art recognize themodified feature as still having the required characteristics andcapabilities of the unmodified feature. In general, but subject to thepreceding discussion, a numerical value herein that is modified by aword of approximation such as “about” may vary from the stated value byat least ±1, 2, 3, 4, 5, 6, 7, 10, 12 or 15%.

All of the devices, systems and/or methods disclosed and claimed hereincan be made and executed without undue experimentation in light of thepresent disclosure. While the devices, systems and/or methods of thisinvention have been described in terms of preferred embodiments, it willbe apparent to those of skill in the art that variations may be appliedto the compositions and/or methods and in the steps or in the sequenceof steps of the method described herein without departing from theconcept, spirit and scope of the invention. All such similar substitutesand modifications apparent to those skilled in the art are deemed to bewithin the spirit, scope, and concept of the invention as defined by theappended claims.

Furthermore, no limitations are intended to the details of constructionor design herein shown, other than as described in the claims below. Itis therefore evident that the particular embodiments disclosed above maybe altered or modified and all such variations are considered within thescope and spirit of the disclosure. Accordingly, the protection soughtherein is as set forth in the claims below.

Modifications, additions, or omissions may be made to the systems andapparatuses described herein without departing from the scope of theinvention. The components of the systems and apparatuses may beintegrated or separated. Moreover, the operations of the systems andapparatuses may be performed by more, fewer, or other components. Themethods may include more, fewer, or other steps. Additionally, steps maybe performed in any suitable order.

To aid the Patent Office, and any readers of any patent issued on thisapplication in interpreting the claims appended hereto, applicants wishto note that they do not intend any of the appended claims to invokeparagraph 6 of 35 U.S.C. § 112 as it exists on the date of filing hereofunless the words “means for” or “step for” are explicitly used in theparticular claim.

What is claimed is:
 1. A method for musical or linguistic learningcomprising: providing a device comprising an input/output interface, amemory and one or more processors communicably coupled to theinput/output interface and the memory, wherein the input/outputinterface comprises at least a display and a speaker; selecting amusical or linguistic work via the input/output interface; retrieving avisual data, an audio data and an animation or video for the selectedmusical or linguistic work from the memory or the input/output interfaceusing the one or more processors; displaying an action/function bar anda navigation bar on the display, wherein the action/function barcomprises a delay data box, a number of repetitions data box and anautoplay button; selecting a portion of the musical or linguistic workusing the navigation bar; displaying a part of the visual datacorresponding to the selected portion of the musical or linguistic workon the display; selecting a portion of the displayed visual data via theinput/output interface; providing an indication on the displaycorresponding to the selected portion of the visual data using the oneor more processors; and playing a portion of the audio data and aportion of the animation or video corresponding to the selected portionof the visual data using the speaker, the display and the one or moreprocessors.
 2. The method of claim 1, wherein: the visual data comprisesa musical score data for the musical work; and the visual data comprisesa text, character, gesture, sign and/or symbol data for the linguisticwork.
 3. The method of claim 1, wherein the visual data and the audiodata are stored together in one or more files or stored separately inthe one or more files.
 4. The method of claim 1, further comprisingwaiting a specified period of time and replaying the portion of theaudio data corresponding to the selected portion of the visual datausing the delay data box, the number of repetitions data box, theautoplay button, the speaker and the one or more processors.
 5. Themethod of claim 4, further comprising selecting the specified period oftime and a specified number of replays via the input/output interface,the delay data box and the number of repetitions data box.
 6. The methodof claim 1, further comprising selecting a musical instrument, a vocalpart or a language using the input/output interface, wherein the visualdata and the audio data correspond to the selected musical instrument,the selected vocal part, or the selected language.
 7. The method ofclaim 1, further comprising providing the indication on the displaycorresponding to the audio data being played using the one or moreprocessors.
 8. The method of claim 1, further comprising recording auser rendition of the selected portion of the visual data using theinput/output interface.
 9. The method of claim 8, further comprisingcomparing the user rendition to the audio data, calculating an accuracyscore for the user rendition, and displaying the accuracy score on thedisplay using the one or more processors.
 10. The method of claim 9,further comprising providing one or more suggestions to improve theaccuracy score on the display using the one or more processors.
 11. Themethod of claim 9, further comprising displaying or highlighting one ormore errors in the user rendition on the display using the one or moreprocessors.
 12. The method of claim 8, further comprising creating auser visual data based on the user rendition and displaying the uservisual data on the display using the one or more processors.
 13. Themethod of claim 12, further comprising highlighting or visuallyindicating any differences in the user visual data and the visual dataon the display using the one or more processors.
 14. The method of claim1, further comprising performing the selected segment of the visual databy a user.
 15. The method of claim 14, wherein performing the selectedsegment of the visual data by the user occurs after or in combinationwith playing the portion of the audio data corresponding to the selectedportion of the visual data.
 16. A device comprising: an input/outputinterface comprising at least a display and a speaker; a memory; one ormore processors communicably coupled to the input/output interface andthe memory; and wherein a musical or linguistic work is selected via theinput/output interface, a visual data, an audio data and an animation orvideo for the selected musical or linguistic work is retrieved from thememory or the input/output interface using the one or more processors,an action/function bar and a navigation bar are displayed on thedisplay, wherein the action/function bar comprises a delay data box, anumber of repetitions data box and an autoplay button, a portion of themusical or linguistic work is selected using the navigation bar, a partof the visual data corresponding to the selected portion of the musicalor linguistic work is displayed on the display, a portion of thedisplayed visual data is selected via the input/output interface, anindication is provided on the display corresponding to the selectedportion of the visual data using the one or more processors, and aportion of the audio data and a portion of the animation or videocorresponding to the selected portion of the visual data is played usingthe speaker and the one or more processors.
 17. The device of claim 16wherein: the visual data comprises a musical score data for the musicalwork; and the visual data comprises a text, character, gesture, signand/or symbol data for the linguistic work.
 18. The device of claim 16wherein the visual data and the audio data are stored together in one ormore files or stored separately in the one or more files.
 19. The deviceof claim 16, wherein a specified period of time is waited and theportion of the audio data corresponding to the selected portion of thevisual data is replayed using the delay data box, the number ofrepetitions data box, the autoplay button, the speaker and the one ormore processors.
 20. The device of claim 19, wherein the specifiedperiod of time and a specified number of replays are selected via theinput/output interface, the delay data box and the number of repetitionsdata box.
 21. The device of claim 16, wherein a musical instrument, avocal part, or a language is selected using the input/output interface,and the visual data and the audio data correspond to the selectedmusical instrument, the selected vocal part, or the selected language.22. The device of claim 16, wherein the indication is provided on thedisplay corresponding to the audio data being played using the one ormore processors.
 23. The device of claim 16, wherein a user rendition ofthe selected portion of the visual data is recorded using theinput/output interface.
 24. The device of claim 23, wherein the userrendition is compared to the audio data, an accuracy score for the userrendition is calculated, and the accuracy score is displayed on thedisplay using the one or more processors.
 25. The device of claim 24,wherein one or more suggestions to improve the accuracy score areprovided on the display using the one or more processors.
 26. The deviceof claim 24, wherein one or more errors in the user rendition aredisplayed or highlighted on the display using the one or moreprocessors.
 27. The device of claim 23, wherein a user visual data iscreated based on the user rendition and the user visual data isdisplayed on the display using the one or more processors.
 28. Thedevice of claim 23, wherein any differences in the user video data andthe visual data are highlighted or visually indicated on the displayusing the one or more processors.
 29. The device of claim 16, whereinthe device comprises an electronic tablet, a mobile phone, a laptopcomputer, a desktop computer, a smart TV, a smart glasses, or a gamingdevice.
 30. The device of claim 16, wherein the input/output interfacefurther comprises a touch display, a keyboard, a mouse, a voice control,a motion control, a wired communications interface, or a wirelesscommunications interface.